To enhance the efficiency of testing, many tools, such as IBM Rational Functional Tester and HP WinRunner, have been developed for automation testing.
FIG. 1 shows a typical flowchart for the process of executing page testing with automation testing tools.
When automation testing is executed with automation testing tools, a series of actions are conducted by the tester under the record mode of the automation testing tool. During such process, the automation testing tool will automatically record testing scripts according to an action conducted by the tester (Step 101) and set a check point (Step 105). The use of a check point is to determine whether a page is successfully loaded or not upon testing this page. It is possible to make such determination by checking a property of HTML predetermined for the page, wherein a conventional property is wait time on loading of a page. Prior to playback of the test scripts, such automation testing tools need to set a wait time value for loading the page (Step 110). When starting to playback the test scripts (Step 115), the wait time functions as a comparing reference: If the page is opened within the wait time (Yes at Step 120), the testing result of this page is identified as successful (Step 125); and if the page cannot be opened within the wait time interval, the testing result of the page is identified as a failure (Step 130).
The wait time for testing all pages is set to a fixed time range in conventional automation testing tools. After the test, if the tester finds some pages cannot be opened within the fixed time range, in the case that the tester thinks the fixed time range is too small or that perhaps the network is congested, etc.), then the tester may manually modify the setting value of the wait time until those pages are eventually opened or the testers assume that those pages are failed in automation result. On the other hand, if the tester thinks the fixed time range reflects the actual condition of the pages (for example, the fixed time range is large enough), or the purpose for the test is just to obtain an approximate result for reference, then the results of these pages may be directly marked as failure.
As mentioned above, in the conventional automation testing, since the wait time of each page to be tested is set to the same fixed value, the testing result will not reflect conditions due to a complicated environment, such as unstable network, different response time to different kinds of pages and middlewares, etc. Specifically, if the fixed wait time for a page is set to a small value, it will affect the accuracy of the testing result. For example, some normal pages which need a longer time range to be opened would be determined as failures because the wait time is set too small. In this case, it is inconvenient that after testing, the tester will frequently and manually adjust the value of the wait time of the pages that can be opened, but are not opened due to the small wait time. On the other hand, if such wait time is set to a larger value to ensure that no page will be missed due to the setting of the wait time, the whole testing process will take a long time and time will be wasted at the pages that cannot be opened at all, causing a lower efficiency of the test.